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Method of processing images 



FIELD OF THE INVENTION 

The invention relates to a method of processing images in images comprising 
curvilinear structures, the method comprising, in parallel, a step of filtering said images and a 
decision step intended to select the pixels of the images pertaining to an interesting 
5 curvilinear structure. The invention also relates to a computer program intended to perform 
this method, as well as to a medical scanning apparatus comprising an image acquisition 
device and a system for processing these images in accordance with said method. The 
invention may be used, for example, in the field of medical scanning. 



1 0 DESCRIPTION OF THE PRIOR ART 

Numerous images comprise interesting curvilinear structures. This is the case 
in medical images, where these interesting curvilinear structures may be blood vessels or a 
catheter. The typical diameter of a catheter varies between 1 and 3 pixels and the typical 
diameter of a blood vessel varies between 5 and 20 pixels. 

15 A system for processing images in a sequence of images acquired by way of 

X-ray fluoroscopy is described in US patent 5,809,105. This system comprises a digital 
image acquisition device, a filtering device using a Gaussian filter intended to eliminate the 
noise caused by the random fluctuations of the electric signal during acquisition of the 
sequence of images, and a detection device intended to distinguish the pixels pertaining to an 

20 interesting structure from pixels pertaining to the basic image. This detection device is based 
on a method of detecting edges, using the gradient and does not allow a good selection of the 
interesting curvilinear structures which are generally difficult to distinguish in a fluoroscopy 
image because the X-rays used have a low power and the image is thus very noisy. 

25 SUMMARY OF THE INVENTION 

It is a first object of the invention to provide a method allowing improvement 
of the selection of interesting curvilinear structures in images. 

According to the invention, a method of processing images as defined in the 
opening paragraph has the following characteristic features. The decision step comprises, in 
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parallel, a sub-step of estimating the direction of each image pixel, as well as a sub-step of 
analyzing the connectivity of neighboring pixels based on their directions at the end of the 
sub-step of estimating the direction of each image pixel, and a sub-step of selecting groups of 
pixels as a function of the result of the sub-step of analyzing the connectivity of neighboring 
5 pixels based on their directions, at the end of the step of filtering. 

An advantage of this method is that it allows elimination of false alarms while 
preserving the interesting curvilinear structures. This method is robust and yields stable 
results in numerous types of images. Another advantage of this method is that it allows 
selection of thick curvilinear structures, which is interesting in medical scanning where 

10 certain structures such as blood vessels are thick curvilinear structures. 

It is a second object of the invention to provide a computer program and an 
image-processing system for performing this method. 

It is a third object of the invention to provide a medical scanning apparatus 
comprising an image acquisition device and a device for performing said method. 

15 The invention takes the following aspects into consideration. The selection of 

interesting curvilinear structures in images is often difficult. This is the case in medical 
images which may be acquired, for example, by way of X-ray fluoroscopy or by way of 
magnetic resonance. 

The following principle allows improvement of the selection of interesting 

20 curvilinear structures in an image. A digital image constituted by a table of values 

corresponding to the grey levels of each pixel is filtered by means of a filter whose role is to 
select a large part of the curvilinear structures in the image. The nature of the filter is not 
critical since it fulfills this role. This filter may also form a threshold of the filtered image in 
order to effect a first selection of the pixels for the purpose of reducing the number of pixels 

25 to be treated in the following steps. This filtering operation simultaneously plays the role of 
the filtering device and of the detection device described with reference to the prior art. 

Parallel to this filtering operation, the direction is estimated for each pixel of 
the digital image. For example, one may measure the tangential direction of a pixel. Indeed, 
the neighboring pixels pertaining to the same curvilinear structure will have neighboring 

30 tangential directions. A direction image comprising, for each pixel, an angular sector number 
corresponding to its direction can be obtained by this estimation of the direction of each 
pixel. An analysis of the connectivity of neighboring pixels based on their directions is 
realized on the direction image, thanks to the angular sector numbers contained in this 
direction image. The object of this connectivity analysis of neighboring pixels is to obtain a 
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connected image in which the neighboring pixels having apneighboring numbers of the 
angular sector are interconnected. Subsequently, groups of pixels are selected on the basis of 
this connected image for the filtered image in order to eliminate or preserve certain groups of 
pixels as a function of an adjustable parameter. As regards edge detection based on a gradient 
5 method, the method described in the invention allows a finer selection of groups of pixels 
pertaining to an interesting structure and thus elimination of a large number of false alarms. 
Indeed, while a gradient method allows selection of all the pixels of an image presenting a 
certain contrast with their nearest neighbors without taking further remote neighbors into 
account, the method described in the invention allows further remote neighbors to be taken 

10 into account, which is important for the detection of curvilinear structures which generally 
extend through a length of several tens of pixels. 

An image is thus obtained in which the interesting curvilinear structures are 
selected. It is important to note that the structures which are interesting in an image will not 
always be the same according to the application. For example, in a medical image 

15 representing a tree structure of blood vessels, the operator may be interested in selecting the 
catheter having a diameter which is smaller than that of the main vessels. The fact that 
adjustable parameters are used, possibly by the operator, allows selection of curvilinear 
structures of different thicknesses in accordance with the application under consideration. 

The invention and additional characteristic features which may be used to 

20 advantage for carrying the invention into effect will be described in greater detail below with 
reference to diagrammatic Figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a diagram illustrating the characteristic features of the invention; 
25 Fig. 2 illustrates an example of a filter for selecting curvilinear structures in an 

image; 

Figs. 3a to 3d illustrate a technique based on the use of a Hessien matrix for 
estimating the direction of each pixel of the image; 

Fig. 4 illustrates an example of the device for analyzing the connectivity of 
30 neighboring pixels, based on their directions; 

Figs. 5a and 5b are block diagrams illustrating an image-processing system 
and a medical scanning apparatus for carrying out the invention. 

Fig. 6a is a block diagram illustrating a technique of detecting artery 
anomalies from a digitized image, which is an advantageous embodiment of the invention; 
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Fig. 6b illustrates an example of computing an artery diameter; 
Fig. 7 is a block diagram illustrating a technique of detecting artery anomalies 
from two different digitized images, intended for carrying out the invention. 

5 DESCRIPTION OF PREFERRED EMBODIMENTS 

Fig. 1 is a block diagram illustrating the characteristic features of the 
invention. A method of processing images comprises a step of filtering realized by a filter 1, 
a sub-step of estimating the direction of each pixel of the image, realized by a direction 
estimation device 2, a sub-step of analyzing the connectivity of neighboring pixels, realized 
10 by a connectivity analysis device 3 and a sub-step of selecting groups of pixels, realized by a 
selection device 4. 

A digitized image NUMI is applied to the filter 1 . A filtered image RFI is 
obtained in which a large part of the curvilinear structures of the digitized image NUMI is 
selected. In parallel, the digitized image NUMI is applied to the direction estimation device 

15 2. A direction image DIRI is obtained, which comprises a set of angular sector numbers 
corresponding to the directions of all the pixels of the digitized image NUMI. The 
connectivity analysis device 3 has the direction image DIRI as input. The operation of this 
connectivity analysis device 3 will be described in greater detail with reference to Fig. 3 . This 
connectivity analysis device 3 allows a connected image CONI to be obtained in which the 

20 pixels of the direction image DIRI are connected or not connected as a function of their 
angular sector numbers. The filtered image RFI is subsequently applied to the selection 
device 4, which allows a selection of the pixels of the filtered image RFI pertaining to an 
interesting curvilinear structure, as a function of the connected image CONI. A processed 
image PRCI is thus obtained. 

25 Fig. 2 illustrates an example of the filter 1 for selecting the curvilinear 

structures in an image. In the example illustrated in this Figure, the filter 1 is used for a pixel 
P of the image NUMI pertaining to a curvilinear structure STR. It should be noted that the 
filter 1 is used in the same way for all the pixels of the image NUMI. Moreover, in the 
description of Figs. 2 to 4, the curvilinear structures which one wishes to select are dark with 

30 respect to the background of the image. A first neighborhood Nl having a height H and a 
length L is considered on one side of the pixel P. A second neighborhood N2 of the same 
dimensions is considered to be on the other side of the pixel P. The width between these 
neighborhoods Nl and N2 is determined by virtue of the adjustable parameters H and L. The 
gap GAP between the first neighborhood Nl and the second neighborhood N2, i.e. the 
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number of pixels between these 2 neighborhoods is chosen as a function of the width of the 
curvilinear structures STR which one wishes to select. The larger the thickness of the 
curvilinear structures STR which one wishes to select, the wider the gap GAP between the 
two neighborhoods Nl and N2. For example, if curvilinear structures STR having an average 
5 thickness of one pixel are to be selected, which would be the case with a catheter in a medical 
image, a gap GAP of the order of two or three pixels between the two neighborhoods Nl and 
N2 would be chosen. If curvilinear structures STR of an average thickness of between 10 and 
20 pixels are to be selected, which would be the case with a tree structure of blood vessels in 
a medical image, a gap GAP of the order of 20 pixels between the two neighborhoods Nl and 
p 10 N2 would be chosen. The principle of this filter 1 is as follows. The sum E u of the grey levels 
51 of all the pixels situated above and below the pixel P is found for the height H in the structure 

E0 STR. For each row of pixels having a length L pertaining to the neighborhood Nl, 

m symbolized by dotted lines, the average Mn is computed for the grey levels of all the pixels 

w of the neighborhood Nl present in this row. For each row of pixels having a length L 

M 1 5 pertaining to the neighborhood N2, symbolized by dotted lines, the average M i2 is computed 
pfj for the grey levels of all the pixels of the neighborhood N2 present in this row. The minimum 

mini of Mn and Me is computed. The sum E v of all the minima mini of the rows of pixels of 
M the length L is formed for the height H. Finally, the difference C between S u and S v is 

computed in order to obtain the contrast at the pixel P. This operation is performed for all the 
20 pixels of the image NUMI and the image RFI is obtained in which the value of each pixel is 
equal to the contrast C. 

The advantage of this filter 1 is that it allows selection of curvilinear structures 
having selected thicknesses and thus elimination of thicker structures. For example, in 
medical scanning, this filter allows selection of a tree structure of blood vessels and 
25 attenuation of thicker structures such as a blood clot. 

Moreover, this filter can perform a first permissive selection of pixels of the 
digitized image NUMI pertaining to an interesting curvilinear structure, in accordance with a 
criterion using an adjustable parameter X. For example, the selected pixels are those having a 
value which is higher than X times the variance of noise in the image filtered by the above- 
30 described device. The parameter X may be chosen to be more or less important in accordance 
with the desired selection degree. The role of this first selection is to limit the number of 
pixels taken into account in the subsequent steps so as to avoid problems of overloading the 
memory cell. 
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Figs. 3a to 3d illustrate a technical example of estimating the direction of each 
image pixel. The method consists of diagonalizing a Hessien matrix HES described with 
reference to Fig. 3d whose diagonal elements are equal to a first value Lxx and a second 
value Lyy and the other elements are equal to a value Lxy . A value Lxx, a value Lyy and a 
5 value Lxy are computed for each pixel of the digitized image NUMI. Fig. 3a illustrates the 
computation of the value Lxx. For a given pixel of the digitized image NUMI, one considers 
a neighborhood N5 5 of 5 by 5 pixels centered around this pixel and a convolution of this 
neighborhood N55 with a mask Mxx is effected. The value Lxx is thus obtained. Fig. 3b 
illustrates the computation of the value Lyy. For the given pixel of the digitized image 
« 10 NUMI, the convolution of the neighborhood N55 with a mask Myy is effected. The value 
u Lyy i s thus obtained. Fig. 3 c illustrates the computation of the value Lxy. For the given pixel 

m of the digitized image NUMI, the convolution of the neighborhood N55 with a mask Mxy is 

2; effected. The value Lxy is thus obtained. Subsequently, the Hessien matrix HES is 

3 diagonalized and the eigenvector U associated with the obtained smallest eigenvalue is 

!=£. 1 5 retained. This vector is the vector of the tangential direction of the pixel under consideration. 
l Jt Subsequently, the direction of this vector is computed, such that the angle obtained is an 

H angle between 0 and 1 80 degrees. This angle is then divided by 1 0 and the result obtained is 

jy[ rounded off to the highest integral. An angular sector number K for the pixel under 

consideration is thus obtained. In this example, there are 18 different angular sector numbers. 
20 It is possible to use a different number of angular sector numbers, for example, by dividing 
the angle of the vector U by 5 instead of by 10. 

Fig. 4 illustrates an example of the connectivity analysis device 3 based on the 
direction. The example considered applies to 6 pixels of the direction image DIRI 
enumerated 1 to 6 (PI, P6) having 6 associated direction vectors enumerated 1 to 6 (Ul, U6) 
25 and 6 associated angular sector numbers enumerated 1 to 6 (Kl, K6). The principle of the 
connectivity analysis is as follows. For a given pixel PN (where N is an integral number 
between 1 and 6) a neighborhood NN extending in the direction of the vector UN is 
considered, over an adjustable length 1. In the present case, two neighborhoods are 
represented, which are symbolized by dotted line ellipses. The neighborhood Nl is a 
30 neighborhood of the pixel PI extending in the direction of the vector Ul. The neighborhood 
N5 is a neighborhood of the pixel P5 extending in the direction of the vector U5. Let it be 
assumed that the pixel PI is the first pixel of the direction image DIRI treated by the 
connectivity analysis device 3. The value of its angular sector number Kl in the direction 
image DIRI is, for example, between 0 and 18. The pixel PI is then labeled with a label 19. 
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All the pixels of the neighborhood Nl having an associated direction vector near Ul will be 
labeled with the same label 19, in other words, they pertain to the label 19. For example, a 
vector UN of a pixel PN of the neighborhood Nl may be considered to be near the vector Ul 
if the angles of these 2 vectors differ by less than 10 degrees, i.e. if the angular sector number 
5 KN and the angular sector number Kl maximally differ by one unity. In the example 
considered, the pixels P2 and P3 pertain to the neighborhood Nl . As the angular sector 
number K2 is equal to the angular sector number Kl and the angular sector number K3 is 
higher by one unity than the angular sector number Kl ? the pixels P2 and P3 will also pertain 
to the label 19. The same operation is performed for neighborhoods of the pixels P2, P3 and 
M: 10 P4. It is then easy to ascertain that all pixels P2, P3, P4 and P5 will pertain to the label 19. 
5 The pixel P6 has an angular sector number K6 which is 6 unities higher than the angular 

sector number K5. Consequently, it will pertain to a label which is different from 19, for 
y example, 20. The operation is thus performed in a recursive manner for all the pixels of the 

22 direction image DIRI, until all the pixels of the direction image DIRI have been labeled. It 

s 1 5 will be noted that once a pixel of the direction image DIRI has been labeled, its label can no 
ry longer change when the connectivity analysis proceeds. The image CONI constituted by 

'f different labels to which the pixels pertain is thus obtained. 

□ The last step of the method consists of selecting groups of pixels in the filtered 

image RFI in order to preserve the interesting curvilinear structures in this image. The 

20 principle of selecting groups of pixels is as follows. The sums \i of the contrasts (in the 
filtered image RFI) of all the pixels pertaining to the same label in the connected image 
CONI are computed. This operation is performed for all the labels of the image CONI and 
the maximum Y of all the computed ji is considered. Only the pixels pertaining to a label are 
retained, such that ji is higher than the quotient of Y by an adjustable parameter M. The 

25 processed image PRCI constituted by pixels of the filtered image RFI, which have been 

retained after the selection of groups of pixels, is thus obtained. The choice of the parameter 
M allows different degrees of selection. Indeed, if M is small (in practice, of the order of 
several unities), fewer groups of pixels of the filtered image RFI will be retained in the 
processed image PRCI. When M is large (for example, of the order of several hundred in 

30 practice) the quasi-totality of the groups of pixels of the filtered image RFI will be retained in 
the processed image PRCI. It should be noted that the selection of the groups of pixels yields 
good results for thick curvilinear structures such as a tree structure of blood vessels, because 
the neighboring pixels pertaining to such a structure generally have the same direction. 
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Fig. 5 a illustrates an image-processing system for carrying out the invention. 
The image-processing system comprises an image-processing device 5 according to the 
invention, a workstation 6, a screen 7, a mouse 8 and a keyboard 9. 

The digitized image NUMI is processed by the image-processing device 5 in 
accordance with the method described in the invention. This image-processing device 5 may 
be implemented as a programmable microprocessor which forms part of the workstation 6 or 
of a circuit comprising elements such as filters, memories or logic operators for realizing the 
different steps described in the invention. The screen 7, the mouse 8 and the keyboard 9 
allow visualization of the processed images and allow adjustment of the different adjustable 
parameters described in the invention. The image-processing device may be connected to a 
storage device for storing the different images obtained during processing. 

Fig. 5b illustrates a medical scanning apparatus for carrying out the invention. 
The apparatus comprises an X-ray source 10, a table 1 1 to carry a patient 12 to be examined, 
an optical system 13 and an image-processing system 14. 

The optical system 13 allows acquisition of digitized images from X-rays 
emitted by the X-ray source 10 examining the patient 12. The image-processing system 14 
may be, for example, the image-processing system described with reference to Fig. 5a. 

Fig. 6a is a block diagram illustrating a method of processing images, intended 
for detecting artery anomalies such as, for example, a stenosis or a diffuse coronary disease. 
Such a method comprises a step of selecting curvilinear structures, realized by the image- 
processing device 5, a step of skeletonizing, realized by a skeletonizing device 15, a step of 
measuring artery diameters, realized by a device 16 for measuring artery diameters, and a 
step of taking decisions, realized by a device 18 for taking decisions on the basis of rules 17 
predefined by an operator. 

A digitized image NUMI, comprising at least one artery, is processed by the 
image-processing device 5. The processed image PRCI obtained is subsequently processed 
by the skeletonizing device 1 5 in accordance with a method known to those skilled in the art. 
A skeletonizing operation consists of making an object thinner while preserving its topology, 
until its width is reduced to one pixel. Such a method is described in, for example, EP 0 840 
552, entitled "Procede de traitement d'images numeriques pour l'extraction automatique 
d'objets en forme de rubans". This method describes steps of extracting median pixels and 
constructing skeletons of objects by means of tracking. An image SKE of the artery skeleton 
is supplied. Based on the processed image PRCI, the skeleton SKE and the direction image 
DIRI, computed by the image-processing device 5, the device 16 for measuring artery 
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diameters computes artery diameters in the following manner, illustrated in Fig. 6b: at each 
point S of the skeleton SKE, one considers the points of the artery situated on the 
perpendicular to the direction of the point S going through S, which perpendicular is obtained 
from the direction image DIRI. On said perpendicular, side points PI and P2 are considered, 
5 which belong to the artery and have at least one directly adjacent neighbor that does not 
belong to the artery, i.e. having a zero value in the processed image PRCI. The diameter of 
the artery is evaluated as the distance between the two side points PI and P2, for example, in 
accordance with the Euclidian standard. A set of diameters DIA of the artery is thus obtained. 
The device 1 8 for taking decisions compares the set of diameters DIA of the artery with 

10 rules 17 predefined by an operator and takes a decision on the presence of possible artery 

anomalies. An image ABM of these possible anomalies is supplied at the output of the device 
18 for taking decisions. 

Fig. 7 is a block diagram illustrating a method of processing images, intended 
for detecting artery anomalies in three dimensions (3D). Such a method allows avoidance of 

15 artery masking such as reductions or overlaps. It has two different images of the same artery 
as input and comprises a step of selecting curvilinear structures, realized by the image- 
processing device 5, a step of skeletonizing, realized by a skeletonizing device 15, a step of 
reconstructing the 3D image of the artery, realized by a device 19 for reconstructing 3D 
artery images, a step of measuring artery diameters, realized by the device 16 for measuring 

20 artery diameters, and a step of taking decisions, realized by a device 18 for taking decisions 
on the basis of rules 17 predefined by an operator. 

Two different digitized images NUMI1 and NUMI2 of the same artery are 
first processed individually by the image-processing device 5 in accordance with the method 
described in the invention. The images PRCI1 and PRCI2 obtained are skeletonized in 

25 parallel by the skeletonizing device 15 and lead to the skeletons SKE1 and SKE2, 

respectively. The images PRCI1 and PRCI2 and their associated skeletons SKE1 and SKE2, 
respectively, are applied to the input of the device 19 for reconstructing 3D artery images, 
which forms a 3D image from at least two 2D images. This device supplies a 3D image 3DT 
of the artery. By way of non-limitative example, such a device may employ the method 

30 described in the reference J. Chen, "3D Reconstruction of Coronary Arterial Tree to 

Optimize Angiographic Visualization", IEEE Transactions on Medical Imaging, Vol. 19, No. 
4, April 2000. The device 18 for measuring artery diameters utilizes this 3D image 3DT of 
the artery together with the tangential direction images DIRI1 and DIRI2, computed by the 
image-processing device 5, for measuring the set of diameters DIA of the artery. Similarly as 
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in the 2D case, the step 1 8 of taking decisions detects, on the basis of the set of diameters 
DIA of the artery and rules 1 7 predefined by an operator, the possible artery anomalies and 
supplies an image ABM giving a precise localization of these possible anomalies. 

The description above with reference to the Figures illustrates rather than 
5 limits the invention. It will be evident that there are numerous alternatives within the scope of 
the appended claims. In this respect, several remarks will be made below. 

There are numerous ways of selecting curvilinear structures in an image. Fig. 
2 only illustrates a possible method in which the filter 1 uses two neighborhoods of a pixel, 
with adjustable dimensions. It is possible to use other filters for selecting curvilinear 
1 0 structures in an image, taking the shape of, for example, masks. 

There are numerous ways of estimating the direction in a given image. Fig. 3 
only illustrates a possible method in which the estimation of the direction of each pixel 
depends on the diagonalization of a Hessien matrix. It is possible to use other techniques for 
estimating the direction of each pixel, for example, by using a discrete channel. 
1 5 There are numerous ways of effecting a connectivity analysis of neighboring 

pixels based on their directions in a given image. Fig. 4 only illustrates a possible method in 
which the connectivity analysis of neighboring pixels based on their directions uses a 
neighborhood of a pixel extending in the direction of the pixel, in which the width of this 
neighborhood is adjustable by means of a parameter 1. It is possible to use other techniques 
20 for performing the connectivity analysis based on the direction, with different neighborhoods. 



